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Introduction 

This action research presents data about an online Master of Arts in Education (MAE) in 
Learning and Behavior Disorders (LBD), P-12 at Western Kentucky University. During 
extensive program revisions, the program developed a collaborative assessment model while 
preparing more than 300 special education personnel for work in rural schools. 

Exceptional Education (EXED) provides a strong field-based, multi-disciplinary, and 
competency-based program that integrates research-based curriculum and pedagogical 
knowledge with practical skills relevant to the targeted student population. The intent is to meet 
the demand for quality professional educators. A primary goal of this preparation program is to 
develop skills and enhance dispositions so that the candidate and their P-12 students can 
experience success in rural schools. 

The online MAE in EXED leads to P-12 LBD certification. The EXED program 
emphasizes culturally sensitive practices for effective teaching in high-poverty, demographically 
diverse rural schools. The MAE requires 30 semester hours, a comprehensive exam, a research 
tool, and a professional development portfolio. LBD certification requires a passing score on 
Praxis II administered by the Educational Testing Service (ETS). 

The collaborative assessment model utilized by this program replaces outdated and 
inefficient structures and processes. This model focuses on preparing and sustaining quality 
professionals, while measuring and documenting both program and effectiveness of student 
learning through the use of technology, data driven decision making, collaborating with multiple 
partners, and coordinating these factors in alignment with Interstate New Teacher Assessment 
and Support Consortium (INTASC), National Council for Accreditation of Teacher Education 
(NCATE), Southern Association of Colleges and Schools (SACS), Association for Educational 
Council for Exceptional Children (CEC) and International Society for Technology in Education 
(ISTE) teacher standards. The ultimate goal focuses on the student’s ability to affect K-12 
learning. Online graduate course content and projects in this program include technology 
integration and assessment of student learning in curriculum applications as well as to enhance 
productivity and professional practices. 

Theoretical Context 

The assessment movement in higher education has evolved into a resignation of dealing 
with political accountability and economic implications while most of us still embrace the 
primary focus and significance of improving student learning. We have shifted from teaching- 
centered to learning-centered where higher education is producing learning rather than just 
providing instruction. (Angelo, 1999). 

If we acknowledge that assessment drives student learning, it will likely remain at the 
center of the curriculum design process, and will be central to the student learning experience. 
(Ramsden, 1992; Biggs, 1999). Higher education instructors need a principled basis for 
designing new forms of assessment, closely aligned with instructional goals and standards while 
employing the interactive features of online technology. (APA, 1993). A constructivist learning 
environment is based on social interaction, communication, exchange of views, collaboration and 
support for learners to take more responsibility for the learning process through learner-centered 
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tasks (McLoughlin and Oliver, 1998). The features of the student-centered curriculum and 
assessment include performance-based tasks that require students to create a product, engage in 
teamwork, and self and peer assessment (Laurillard (1996). 

Reeves (2000) suggests three main strategies to integrate alternative assessment into 
online learning environments: cognitive assessment, performance assessment, and portfolio 
assessment. Further, he proposes five critical aspects of performance assessment. These are 
focused on complex learning; engagement in higher-order thinking and problem solving skills; 
stimulation of a wide range of active responses; involvement with challenging tasks that require 
multiple steps; and significant commitment of student time and effort. Simonson et al. (2000) 
claims that proponents of alternative, performance based assessment suggest that the content 
validity of authentic tasks is ensured because there is a link between the expected behavior and 
the ultimate goal of skill/learning transfer. 

Higher education faculty need to develop a learning community culture. Four 
preconditions are critical to this collective culture. First, we need to develop shared trust by 
highlighting individual successes and helping faculty members feel respected, valued, safe, and 
in the company of worthy peers. Second, a faculty can share vision and goals by collectively 
identifying learning-related goals worth working toward and problems worth solving while 
considering the costs and benefits to faculty members and students. For example, a simple 
approach may be to ask faculty to list two or three assessment questions they would like to see 
answered in the coming year or things they would like to ensure that students learn well before 
graduating. Then, common goals are identified across the lists. When common goals are 
determined, they must clear, specific, linked to a timeframe, feasible, linked to standards, and, 
most importantly, significant to the field. Third, a shared language or concepts must be built. 
Before a faculty can collaborate productively, they establish common definitions for terms such 
as learning, community, improvement, productivity, and assessment. Fourth, shared guidelines 
must be developed. In other words, build a list of research-based guidelines for using assessment 
to promote student learning and program improvement. Examples of guidelines include engaging 
actively in students’ academic work, setting and maintaining realistic high expectations and 
goals, providing regular and specific feedback, and providing connections of research findings to 
authentic real-world applications of assessment projects. (Angelo, 1999) 

Research Methods 

The purpose of this descriptive, developmental research was to investigate the current 
status of the graduate EXED program to describe "what exists" with respect to three variables — 
student assessment, graduation rates, and collaboration.. The scope of this research was not only 
concerned with the existing status and interrelationships of the three variables but the course and 
program revisions that took place over the last three years. One type of developmental research is 
“Model or System Development” which is the creative development of a model or system 
(paradigm) based on a thorough determination of the present situation or system and the goals 
sought. (Key, 1997) The development of a Collaborative Assessment Model is the primary 
outcome of this three-year research. 

In this section program revisions will be described and then the collaboration between 
faculty will be explained. WKU faculty began their investigation, leading to program revisions 
that uniquely address how quality and capacity will be ensured through research-based pedagogy 
that incorporates the critical components of theory, demonstration, guided practice, and authentic 
application in school and community-based experiences. The research questions used by both 
programs during this reform process are as follows: 
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1. How do we prepare and sustain quality professionals? 

2. How do we measure and document effectiveness of programs? 

3. How do we effectively prepare students in the use of data in decision making? 

4. How do we effectively prepare students in the use of technology in data management? 

Exceptional Education 

The EXED program faculty began their program review with alignment of courses with 
state and national standards — the KETS, NCATE, SACS, CEC, and ISTE. The next step 
involved the creation of a chart outlining all EXED course objectives, assignments, and field 
experiences. At times, faculty members were surprised at the results of this chart. Areas of 
duplication were discovered and negotiation for the appropriate placement of some assessments 
ensued. For example, case studies were required in several courses. Discussion revealed that one 
professor was only requiring this assessment because the instructor felt that students should 
know how to use a case study. Negotiation involved a discussion of the best placement of the 
assessment measure. 

Critical performances are specific assessments which provide evidence about what 
teacher candidates must know and be able to do at different levels of growth and development 
toward one or more teaching standards. They are usually a culminating project of multiple parts 
which encompasses most content and accomplishments in the course. 

Multiple sources of input were utilized to outline critical performances for the overall 
program and individual courses. The critical performances are the result of contributions from 
the EXED Advisory Council concerning necessary and practical skills for teachers, faculty 
expertise, students, graduates, current research and university practices (professional portfolios 
and Teacher Work Samples). Once the members of the faculty identified critical performances, 
they worked on the specific requirements of each critical performance and scoring rubric. 

Another major program revision included the EXED Comprehensive Exam. The old style 
for this exam was a Praxis-like multiple choice exam that had been created by the faculty. An 
item analysis was conducted to determine which courses were represented and which questions 
were most answered incorrectly by students. Not only did this analysis reveal many 
inconsistencies in the exam, but many of the standards and critical performances were not 
represented. A new essay-type of exam was created with a question from each course that was 
correlated to critical performances and standards. Students answer three questions by selecting a 
question from a group of four. Additionally, a performance task was included. 

The development of a graduate survey and database of graduates added an ongoing check 
and balance to the process of continual program assessment. Graduates provide input about their 
preparation to become a special education teacher and suggestions for improvement in the 
program. This graduate survey data, student performance on critical performances in courses, 
EXED Comprehensive Exam passing rate, and Praxis passing rates are analyzed each year and 
used to make adaptations to the program. 

Faculty have developed a new program model, the MAE in LBD, P-12, that increases 
both the capacity and quality of teachers while helping graduate students from underrepresented 
populations to overcome barriers to participation in the LBD program. This new model has 
improved the capacity of the program by implementing strategies to serve students for whom the 
program is currently inaccessible, including students who are employed and unable to enroll in a 
full-time program, students who are not able to commute to campus, students who can not afford 
tuition, and students who have difficulty negotiating barriers to participation due to disability. 
Structural improvements to increase the responsiveness of the LBD faculty to the needs of these 
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diverse students include the use of on-campus programs, interactive distance education 
technology and course delivery, and on-line web delivered courses. WKU continues to develop 
a comprehensive program that allows for maximum accessibility for students. 

The quality of the MAE program has also been improved in several ways. The EXED 
faculty has implemented a number of strategies and activities to make the program more field- 
based, multi-disciplinary and competency-based. The specific revisions and enhancements 
include: (a) the use of cohort groups to facilitate the growth of peer support and collegiality; (b) 
more intense and extensive field-based activities and critical performances that focus on 
culturally competent teaching, including projects requiring multidisciplinary collaboration and 
practical projects with a direct impact on participants’ schools and K-12 student achievement; (c) 
development and expansion of the Professional Development Networks, which included 
trainees’ cooperating/mentoring teachers in the trainees’ field placements, members of advisory 
councils, parents and advocacy groups, and departmental faculty; and (d) restructured internships 
and classroom experiences to assure that competent teachers are trained, who will continue to 
serve students and not leave the field in three to five years. 

Collaboration 

There are many unique features of this program that enhance collaboration. EXED 
graduate programs produce more graduates than any other graduate programs in the university. 
The mode of delivery for is primarily online. Program faculty develop their own content and are 
using high tech systems to create this content (Tegrity, streaming servers, original CD’s, DVDs, 
Blackboard, etc.). 

Collaboration is the key to this descriptive, developmental research. The authors have 
been discussing online assessment strategies for three years. Performance based assessment 
consists of a student’s active generation of a response that is observable either directly or 
indirectly via a permanent product. Performance based assessment must be clearly aligned with 
what has been taught; scoring criteria or rubrics must be shared prior to students working on the 
task; be clearly aligned with standards and objectives and give several models of acceptable 
performances; and encourage student self-assessment and reflection. (Elliott, 1995). An 
assessment is authentic when the nature of the task and context in which the assessment occurs is 
relevant and represents “real world” problems or issues. (Elliott, 1995). 

Performance based, authentic assessment strategies utilized include critical 
performances, teacher work samples, professional portfolios, case studies, annotated 
bibliographies, discussion boards, guided research, webquests, group critiques, interviews, 
surveys, oral presentations via videotape, online tutorials, and online exams. 

Collaboration and discussion between faculty have illuminated the changing roles of 
both faculty and students. The primary responsibility of learning has shifted from the teacher to 
the student. The role of instructor for online courses has become one of intense preparation prior 
to the beginning of class. The instructor provides content, online lectures, structure, assignments 
and assessments linked to standards, sample projects, and schedules. When class begins, students 
must take responsibility for their own learning and, in fact, tailor learning for themselves by 
engaging their individual temperament, circumstances, needs, tastes, and ambition. Students 
have the potential to utilize every aspect of their lives — work, leisure, personal relationships, 
community activities, and course work — to enhance performance on the open-ended, authentic 
projects in each course in the program. The instructor provides support and guidance through 
constant communication (email, announcements, or phone), specific and timely feedback, and 




5 



providing a social context for the class. Each course contains an open discussion board called the 
“Water Cooler” where students and the instructor can discuss any aspect of the course. 

Faculty members have compiled several data collection and analysis tables. Alignment of 
courses with state and national standards was a major first step in program revision. Each course 
includes the standards alignment for that course in its syllabus. Tables representing alignment of 
the objectives, assessment strategies, field experiences, and critical performances for each course 
were constructed. These tables allowed the program faculty to review, compare, and contrast 
assessment strategies, data sources, and standards alignment more effectively. 

All of the K-12 public schools participating with Western Kentucky University in this 
program spent time discussing the issues among their faculty and with public partners. Their 
continual input assists in program design, involvement, and program assessment. Numerous 
efforts are made to meet the needs of the students. Course and program sequences have been 
arranged so part time students can complete their programs in a timely manner. All courses are 
offered online. Tuition assistance is available through grant support, discounts for school district 
partners, and financial aid counseling. For students who have barriers to participation due to 
disability, accommodations are made through assistance from student support services and using 
multimedia experiences that are inclusionary. 

Field experiences in the form of hands-on or field-based application projects are a part of 
most courses in the program. The primary clinical or practicum field experience is in EXED 
590. For the EXED program this semester course follows the Kentucky Teacher Internship 
Program cycle guidelines and requires site visits by practitioners, development of a professional 
growth plan (PGP), videotapes of teaching accompanied by lesson plans and interventions that 
demonstrate mastery of identified concepts on the PGP, and a teacher work sample. 

Several assessment strategies provide students with experience in using data in decision 
making. The teacher work sample requires creating and teaching a unit of instruction. Students 
use assessment data to profile and analyze student learning and communicate information about 
student progress and achievement. The reflection and self-evaluation section requires students to 
analyze the relationship between his or her instruction and student learning in order to improve 
their own teaching practice. Case studies furnish students the opportunity to analyze authentic 
situations and provide solutions incorporating theory and practice. The action research project 
requires students to conduct research in their own classroom, library, and/or technology center, 
analyze the data, and write a journal-type article. (Oberg & McCutcheon, 1987). Field 
experiences afford students the opportunity to make decisions about interviews, on-site visits, 
collaboration, and field projects. The professional portfolio consists of student work that displays 
mastery of standards; a purposeful collection of student work that exhibits the student’s efforts, 
and evidence of student reflection. (Bailey, 1998). 

Use of technology to manage data is demonstrated by instructors and students. The 
authors use Tegrity or Camtasia to create demonstrations, teaching or lecture videos that are 
either available to students on a streaming server, on a CD or DVD distributed to students. The 
authors use BlackBoard to hold online discussions, build a learning community, and post grades . 

Students use technology in three ways. One way is the use of technology to actually take 
the course — use of Internet to participate in the course site in BlackBoard, email, use course 
CDs, use software to create projects, etc. The second use of technology is learning to integrate 
technology in instruction and student learning. When technology integration is a focus, use of the 
technology and its integration is in the scoring rubric for the project. Third is the use of 
technology by students to manage data. Students learn how to create a spreadsheet to record 
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pupil assessment data and create charts for the teacher work sample. Students leam how to create 
a database on instructional topics and create specific types of questions to stimulate higher order 
thinking skills. 



Results & Conclusions 

Results and conclusions are presented below with the research questions: 

1. How do we prepare and sustain quality professionals? 

a. Since the EXED MAE began in June 2002, 405 students have been admitted and 
155 have already graduated and are fully certified teachers. Four students 
dropped the program for various reasons. Of the 250 continuing, 75 are 
scheduled for graduation summer 2006, with the remaining anticipating 
graduation in 2007 or 2008. All students have passed the comprehensive exam 
and completed the Teacher Work Sample. The initial PRAXIS pass rate for the 
program is 94%. The remaining 6% pass when they retake the exam. 

b. Multiple forms of performance-based, authentic assessment provide a more 
accurate picture of student achievement as well as significantly increase the 
quality of graduates in K-12 settings. Shared assessment strategies include 
professional development in schools, authentic assessments, action research 
projects, problem based learning, simulations, case studies, web enhances 
instruction, electronic, standards-based professional portfolios, etc. 

c. Faculty and school practitioners have engaged with the Renaissance Project to 
employ the teacher work sample to showcase the effect teacher candidates have 
on K-12 student achievement. 

d. The changing faculty and student roles in these models are discussed and 
documented. The instructor’s role is one of intense preparation prior to 
commencement of a course and switches to more of a role of support and 
guidance once the class begins. The student is an active learner responsible for his 
or her own learning. 

e. Objectives, content, and assessment are aligned vertically with course objectives, 
content and assessment and horizontally with state, national, learned societies, 
and accreditation standards. 

f. The authors model effective technology integration techniques by developing 
slideshows with course content, teaching/demonstration videos, and creating 
course “text” CDs. 

2. How do we measure and document effectiveness of programs? 

a. Critical benchmark measurements are presented in Table 1 documenting the 
performance and effectiveness of students in the program. Data is gathered at 
admission, during courses, during field experience, exit data, and follow-up. 

b. Programs meet NCATE, SACS, CEC, ISTE and KETS standards. 

c. Program delivery options in the EXED program include online, off campus, 
cohorts, and distance education. 

d. Field experiences are necessary to provide the authentic context for the 
acquisition and demonstration of performance standards. Performance 
measurement includes instructor observation, videos of teaching, supervisor 
assessment, and student self-evaluation and reflection. 
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e. Graduate surveys of new graduates and graduates after their first and third year of 
employment in the field are aligned with state teaching standards and dispositions. 
This provides valuable data about preparation for job success and suggestions for 
improvement. 

f. Program decisions are focused on increasing student achievement, not high stakes 
testing results. 

3. How do we effectively prepare students in the use of data in decision making? 

a. Projects and assessments demonstrating the effective use of student data are 
shared for the program. Examples include action research projects, teacher work 
samples, case studies, and professional portfolios. 

b. Field experiences provide students the opportunity to make decisions about 
interviews, on-site visits, collaboration, and field projects. More importantly, 
students make everyday decisions in authentic teaching situations with real 
children. 

4. How do we effectively prepare students in the use of technology in data management? 

a. Online instruction is a valuable part of the program. Students must use technology 
to manage data in each course. 

b. Students must decide how to integrate technology in instruction. When 
technology integration is a focus in a course, the use of the technology and its 
integration are in the scoring rubric for the project. 

c. Students leam to use technology to manage data through spreadsheets, databases, 
and statistical applications. 

The WKU Exceptional Education program is based on a common conceptual framework 
while the contextual experience of each student is slightly different. Emphasis on quality is a 
constant. Faculty have truly shifted from a teaching-centered to a learning-centered model as 
described by Angelo (1999). A learning community culture among faculty and students is 
developing as a result of this collaborative model. A shared trust is evidenced through the mutual 
respect and collective efforts in program revisions. Faculty have developed shared visions, goals, 
and language through a revised conceptual framework. The authors have developed shared 
guidelines for promoting performance-based, authentic assessment to strengthen student learning 
and program improvement. 

Professional education is a continuing process beginning with, not ending with, initial 
preparation. Strong content expertise is required of all teachers. You cannot teach what you do 
not know. Attainment of program objectives requires a specific learning sequence — the 
acquisition of knowledge, the development of skills, and controlled functional use of skills. 

Implications for Practice and 
Recommendations for Further Research 

This research goes to the very heart of professional education preparation in higher 
education today. Colleges and universities face increasing demands and many of the systems 
and structures currently in place will not meet future needs. This research focuses on enhancing 
present preparation options, developing additional options, collaborating with multiple partners, 
and coordinating all these in alignment with state and national standards, while focusing on the 
student’s ability to affect K-12 learning. 

The authors plan to continue a longitudinal study of graduate performance, revise 
graduate surveys to reflect how practice relates to state and national standards in their jobs, how 
well their graduate program prepared them for their jobs, and suggestions for improvement in the 




programs. The authors plan to gather this data from graduates immediately after graduation, one 
year later, three years later, and five years later. Such hard data will be invaluable in fine tuning 
and reporting effectiveness of the program. 
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